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ABSTRACT 


A new technique for the adaptive estimation of non-stationary 
statistics necessary for Bayesian classification Is developed. The 
basic approach to the adaptive estimation procedure consists of two 
steps: (1) an optimal stochastic approximation of the parameters of 

interest and (2) a projection of the parameters in time or position. 

A divergence criterion Is developed to monitor algorithm performance. 
Comparative results of adaptive and non-adaptlve classifier tests are 
presented for simulated four dimensional spectral scan data. 

This work was supported by NASA contract NAS8-30878. 
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6 n c true value of distribution parameter at time (position) n. 

Y p = data sample classified into a particular class at tine (position) 
n. 

= "refined" estimate of 0„ made after classification number n 
n n 

provides new data sample. 

X = "projected" estimate of o -j made at preceding time (position) n. 
e p = mean square error (X n - e n+ ^) . 

Y n _i = weight used in stochastic approximation to specify weighted 

average of past estimate and present data (chosen to minimize 

2 

mean-square error, e n ). 

— y ? 

E„ = mean square error (X„ - 0 1 . 
n n n n 

2 2 
N(u, a ) denotes normal distribution of mean u and variance a . 




lx 


CHAPTER I 


INTRODUCTION 

A Bayes classifier for M pattern classes is essentially a 
mechanization of M discriminant functions of the patterns These 
functions ^re of the form 

d.(x) * p(x/w 1 ) p(u> 1 ) (1) 

1 * 1, 2, .... M 

where p(x/u^) is the probability density function of the patterns of 
class to., and p(u^) is the a puo-u probability of this class, that is 
the probability of occurrence of class it.. The maximum discriminant 
function will correspond to the minimum conditional risk. In other 
words, the Bayes classifier will minimize total expected loss, where 
loss represents classification error [1], 

In order to make a decijlon on a particular pattern x, the 
classifier computes d-j(x), d 2 (x), ...» d fl (x), and assigns x to 
class uj if dj(x) has the largest value. Ties are resolved arbi- 
trarily. Because the Bayes classifier has found such wide acceptance 
in pattern recognition, this classifier will serve as the basis for 
an adaptive recognition system capable of adjusting itself to a 
changing environment. 
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The structure of a Bayes classifier is determined primarily by 
the conditional densities p(x/w,). Of the various density functions 
that have been investigated, none has received more attention than 
the multivariate normal density. Although this attention is due 
largely to its analytical tractability, the multivariate normal density 
is also an appropriate model for an important situation: the case 

where the feature vectors x for a given class represent a single 
typical or prototype vector , mildly corrupted by zero mean sampling 
and measurement noise [2,3]. 

For M pattern classes the general multivariate normal density 
functions may be written as 


p(x/u>.) 


1 


-1 


(2")" /2 |C 1 


■jjj exp[-l/2(x-u 1 ) C." (x-^)] 


( 2 ) 


1 = 1,2 M 

n = dimensionality of x 


where 


Mi s 


(3) 


» 


and 


C 1 = EKx-^Hx-u,) ] . 


(4) 
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For class aw the Bayes decision function which minimizes probability 
of classifier error Is found to be d.j(x} ■ n(x/u)^ )p(u>^ ). Due to the 
exponential form of the normal density function, it Is more convenient 
to work with the natural logarithm of this decision function [1], 

The decision function may therefore be written as 


d^x) - Ln p(x/u 1 )p(a) i ) = Ln p(x/o> i ) + Ln p(u i ) (5) 


1=1,2, ...» M. 


Dropping the term n/2 Ln (2 ti ) because it is common to all M decision 
functions being compared yields 


d.j (x) = Ln p(u.j) - l/2Ln|C i | - l/2[(x-i^) C^" 1 (x-u 1 )] (7) 

1=1,2, ..., M. 


An examin 3 tion of Equation (7) reveals that the changing 
environment to which the system must adapt is composed of the parti c- 


* 

ular class mean vectors u. and covariance matrices C^. In the context 
of a classification, to adapt means to provide the classifier optimal 
current estimates of parameters necessary for the classification. 

The parameters may vary with time or position. 

In this thesis various stochastic approximation techniques are 
presented for adaptive estimation. A criterion is also suggested 
which nay be used to detect the divergence of estimates of means. 
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The ability of these algorithms to accurately estimate the varying 
mean cf a normal density has been tested by computer simulation. 

These algorithms have been Incorporated Into a Bayes classifier 
to make it adaptive. Comparisons of the various adaptive classifiers, 
Incorporating different estimation algorithms, to the ordinary (non- 
adaptlve) Bayes classifier have been mode revealing the desirability 
of adaptive recognition capability. 

A practical application which has been Implemented in this 
work is real-time classification and physical class boundary definition 
of synthetic multlspectral scan data. These boundaries are those 
between classes in a truth table, and should not be confused with 
Bayes decision surfaces in pattern space. 

The classifier developed here is to servo ?s a model or proto- 
type; therefore, only the two class recognition problem has been 
considered. Extension to the more general multiclass case Involves 
no more difficulty than would be involved ith an ordinary Bayes 
classifier, once the stochastic approximation procedures are under- 
stood. 

The data used In testing the classifiers was generated on the 
IBM 360/65 computer system [4]. Algorithm checkout and classifier 
testing have been performed on the IBM 360/65 and the PDP 11/40 
computer systems. ResulU of classification and subsequent boundary 
definition have been displayed via the Data Disk video system In 
conjunction with the PDP 11/40 computer. 


CHAPTER II 


ESTIMATION ALGORITHMS 

A typical sequence of events for classifying and subsequently 
estimating class statistics at the next time, assuming current 
estimates have been made, is as follows. 

Step 1. The current data sample Y n is classified into a 
particular class using current estimates of parameters for all classes. 

Step 2 . ,1 refined" estimate of the parameters of the class 

chosen in Step 1 is computed by stochastic approximation as 


8 v 


X n-l + Y 


n-l<V X n-l> 


This step is omitted for all other classes for lack of data Y . 

★ 

Step 3 . A "projected" estimate of e n+ -| , X n , may be made by 
transforming X n according to the way the algorithm assumes e is 
changing with n. If the change is due to time, this step is made for 
all classes; if the change is due to position (i.e., as when clas- 
sifying pixels of a multispectral scan frame) within the current class 
being scanned, this step Is performed only for that class chosen in 
Step 1 above. 

Step 4 . Increment n by 1 and return to Step 1. 
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Several notable contributions have been made to tne problem of 
estimating the parameters for a classifier where the class statistics 
vary with time or space J One such adaptive estimator gave larger 
weight to more recent samples, as specified by an empirically 
determined exponential weighting parameter; the consequent "limited 
memory" made the resultant average more up-to-date [5], Intuitively 
the resulting estimates of parameters would be better than an 
unweighted average. Another adaptive estimation algorithm "projected" 
t.he current estimate to the next step by adding an amount of a certain 
form of anticipated change to the last estimate, and then combining 
It with the next data sample in a weighted average with weights 
chosen to minimize the mean square error [6]. This algorithm will 
subsequently be referred to as the CF algorithm, after the authors. 

The algorithm developed in this work consists of "refine" and "project" 
steps [7]. This algorithm differs f n)m the previous one in the 
sense that the former (1) makes projections suitable for more complex 
variations with time, and (2) is arranged to operate as part of a 
Bayes classifier. It will be seen that in both these algorithms the 
"refine" step of combining previous estimate and new data is in the 
form of a stochastic approximation formulation shown previously in 
Step 2 of the typical sequence of events for adaptive classification. 

The CF algorithm is essentially a two step algorithm designed 
to optimally estimate present values of interest rather than to project 


^Time will henceforth denote true time or space (positional 
index), unless otherwise specified. 
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an estimate for future use. The two CF steps are defined as follows: 

(1) *Vl = [1 + t"- 1 )' 1 ! Vl (8) 

«> V x Vl + Vl<V x *n-l> (*) 

★ 

where X n _j represents a projected parameter estimate, X p represents 
the previous estimate of the parameter, e n> and Y p is the current data 
sample. y n _i Is a sequence of positive numbers satisfying the 
conditions of Dvoretzky [8] 


Llm 

n-H» 


r n-l 


= 0, 


£ Y n-1 
n=l 


r 2 

^ Y n-1 * 
n=l 


(10) 


and chosen to minimize the mean square error of the estimates. Because 
this algorithm is similar to the form required by an adaptive Bayes 
classifier, the Incorporation of the technique in a classifier is 
justifiable. In contrast to the empirically derived algorithm 
discussed previously, this procedure produces optimal estimates. 

Examination of equation (8) reveals that this technique assumes 
the estimated parameter to be time varying in a linear or nearly 
linear fashion, with zero initial value. The algorithm lacks com- 
pensation for an initial non-zero offset or bias of the parameter 
value. 


A modification to the algorithm consisted of subtracting the 
initial parameter value from the classified sample, applying the CF 
algorithm to the result, and adding back the initial value to the 
algorithm estimate. In effect the modification allov/ed the algorithm 
to project estimates as if the initial value were zero. 

The algorithm developed here produces true distribution param- 
eter estimates for the class of interest at the next classification 
time. A "refine" step is made, then a "project" step is also made to 
the next time, because once the data has been classified, the classifier 

will require an estimate of the future parameter value, not the present. 

* 

An optimum compromise between the present parar.eter estimate X .j, 
made at the previous step n-1, arid the present data sample Y n is 
made by the stochastic approximation in the "refine" step. The 
"project" operation then provides the classifier an estimate of the 
mean for the time when it is actually needed by the classifier. 

An input, unbiased by variation, is also provided for the next 
stochastic approximation by the "project" step. Therefore, the 
"project" operation should remove (in a statistical sense) the 
estimation bias. 

A name considered appropriate for the algorithm is "polynomial 
fit," hereafter to be referred to as the PF algorithm. The particular 
algorithm presented was derived to make nonlinear estimates of degree 
two ; however, PF actually represents a class of algorithms derivable 
for any finit" degree. The second degree PF algorithm can be specified 
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I t 


The refine step (Step 2 of typical classification sequence) is 


denoted 


X = X* , + y r , (Y-X* ,) = 0 n 
n n-1 n~ 1 n n- 1 n 


( 11 ) 


and the project step (Step 3 of typical classification sequence) 


X* = X + S = 
n n 


°n+l 


( 12 ) 


where 


0 = true value at step n 

n 


and 

S = {[1(1+1 ) - j(j + l)] Y p - [i (1+1 )] Y n-J . + [j ( j+1 )] Y n _ , )/i j(i-J) 


n+1 


(13) 


W 


and 


r n-l 


T 2 

e n - K l°n 

7 ? 2 

e n + °n 


(14) 






1 


10 

and the estimate of mean square error for use in the calculation of 


c n+l ? E (X n ” °n+l )2 


77 2 

. 6 n °n 
T . 2 

e +o 
n n 


2 .„ 2 > 2 


(yi) + (k 2 c +k 3 m c n 


05) 


the required terms for error calculation being 


it! 


k 2 = ' iTT-TT 


(16) 


and 


„ _ i+1 

k 3 ■ ttht 


07) 


with K, defined as the sum of these two or 


K 1 E K 2 + K 3 * 


(18) 


Here the variance of the density function from which samples Y are 

n 

2 

drawn is represented as c n . 

Another form of the PF algorithm has been developed using 
previously projected estimates rather than previous data samples to 






» 
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fit the polynomial assumed In derivation. This form of second degree 
algorithm may be specified as follows. 

The refine and project steps are specified exactly as In 
equations (11) and (12) except with 

s = {[1(1+1) - j(j+l)] X n - [1(1+1)] X n+j + [j(j+l )] X n-1 )/1Ji1-J) 

( 19 ) 

and 



and the estimate of mean square error for use in the calculation of 
15 

v7= (' V ^ + 1^ 2 ^7 + ' 2i ) 



( 22 ) 


I 




r 


12 

and the required constants for error calculation being 


and 



(23) 


(24) 


and with again defined as the sum of these two or 

K 1 = K 2 + K 3 . (25) 

The "project" operation of equation (12) takes a form suitable 

for the manner in which the mean Is assumed to vary with time while in 

★ 

the CF algorithm, "projection" is accomplished as X n = (1+1/n) X n> 

A 

S of equation (12) is an estimate of anticipated change on the next 

time interval based on the assumption that the true value varies as a 

second degree function of time, which is in turn estimated by the 

values of Y . Y„ . , and Y . , or X , X , , and X„ Equations (13) 
n n-i n-j n n-i n-j 

— 1 

or (20) give the optimum weight y p _i to minimize e p+ i for the two 

★ 

forms of the algorithm. The classifier then uses X n as the best 
available value for e + i for the next classification, at step n+1. 

Tests of both forms of the second degree PF algorithm revealed 
that each made equally reliable projections. A disadvantage of the 
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second form Is that approximately twice as much memory Is required In 

order to accomodate all previous estimates of the necessary error 
— 7 

term , 1 * 1 , 2, ... , n. 

The ability of the CF and PF algorithms to "track" the varying 
mean of a Gaussian density has been tested by computer simulation. 

The data { Y n } were drawn from a unit-variance, one dimensional Gaussian 
density with mean 9(n-50) /2500 + 1 for n = 1 to 100, and the algo- 
rithms produced up-to-date estimates of this mean. Ten statistically 
Independent runs were made for 1 < n < 100; the CF algorithm performance 
Is shown in Figure 1, while the second degree PF algorithm performance 
Is shown in Figure 2. For the sake of comparison the performance 
shown in Figure 3 is that resulting from a least mean square error fit 
of a second degree curve to the set { } , K = 1, 2, ..., 100. 

Both the PF and CF algorithms have been applied to the problem 
of adapting to changing mean vectors and covariance matrices of normal 
class signatures. In order to adapt to changing covariance matrices, 
the problem addressed was that of estimating elements of the cor- 
relation matrices separately from the elements of the mean vectors and 
then combining these to form the particular covariance matrices [9]. 

A problem initially encountered using both algorithms was that of 
maintaining a positlve-semidefinlte covariance matrix. 

Based on the assumption that covariance terms vary at a slower 
rate tnan mean vector components, satisfactory estimates of the 
covariance matrices of M classes may be obtained by updating the 
class covariance matrix when P samples have been classified as members 
of that class in the following manner. 
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Figure 1 . Performance of CF (Chien and 
Fu) algorithm. 
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Figure 3. Performance of estimator operating 
as a least mean square error curve fit. 
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Step 1 Involves specifying the Initial covariance matrices for 
M classes C^, 1 = 1, 2, ..., M, a zero matrix , 1 = 1,2, ...» M of 
equal dimensionality to the C matrices, and a counter , 1 = 1, 2, ..., 
M. Each counter should be Initialized to zero. 

In step 2 specify the number of samples P (where P > dimen- 
sionality of pattern vectors) to be used In producing new estimates of 
the covariance matrix of each class.. 

At $Up 3 classify a pattern using the covariance estimate , 

1 8 1, 2, .... M for the classification. 

During step 4, If the pattern was classified Into class j, 
update <j>j according to the relation 

♦1(^+1) ■ t N i + N i W m i 

+ Y(N.+1) Y'(N.+1) (N, m. (N. ) 

1 1 ( N 1 +1 ) 1 1 1 

+ Y(N.+1)] + (N i m 1 (N 1 ) + Y(N i +l )' (26) 

where m^ represents a mean estimate. 

Step 5, Increment the counter Nj by one. 

Step 6, If llj Is less than P, go to step 3, otherwise, go to 

step 7. 

At step 7, replace Cj by Reset ♦j to the zero matrix and 
rezero the counter Nj. Go to step 3. 


P Is chosen greater than the pattern dimensionality In order to 
Insure that the estimate of the covariance will possess an Inverse 
given that the samples are drawn from a rormal population [1]. 
Justification of equation (26) Is given In Appendix A. 


CHAPTER III 

A DIVERGENCE CRITERION 

A problem associated with the CF algorithm and also the PF 
class of algorithms is that their derivations assume the parameters to 
be estimates /ary as some finite degree function. A PF algorithm of 
very high degree, and hence great flexibility is cumbersome to derive 
and to run; likewise, computer execution time increases as the degree 
of algorithm complexity is increased. If the parameter being estimated 
changes with time in a way more complex than assumed by the algorithm, 
the predictions of stochastic approximation techniques may diverge. 
Although the "weak memory" inherent in stochastic approximation will 
compensate somewhat for this problem, it would be desirable to more 
strongly limit the memory by restarting the algorithm at the point of 
divergence, resulting in a piecewise implementation of an estimator. 

A technique for detecting divergence and restarting the particular 
stochastic approximation algorithm in the area of divergence is 
necessary. This restart capability should be provided external to 
the function of the particular stochastic approximation technique 
being implemented. In other words, what is needed is a "monitor" for 
the operation of the algorithm. . 

Consider the problem of the estimation of the unknown mean of 
some distribution Y ~ N(e(n), o ), wtiere the mean 0 (n ) varies with 
time. To assume that this function e(n) might be approximated by 
segments would not be unreasonable. The quantity X n will be considered 
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the approximation to o(n) made by a stochastic approximation algo- 
rithm. Associated with each time Interval Is a random variable Y 
2 th 

with variance a . Tht n in sample value of Y shall be referred to as 
y n . If the particular stochastic approximation algorithm accurately 
estimates e(n) based on y n in some region, it is then possible to 
define a new, time Invariant random variable Z - N(u , o ), where the 
samples z n are given by 


z 


n 




(27) 


However, If x R Is an accurate estimate of e(n), It is clear that u z 
will be zero. A statistical Inference built around the notion of a 
"confidence interval" for a known statistic of the distribution 
function Z may now be made [10,11]. Let the average value of Z be 
calculated by the algorithm 



n 


i <v x i> • 

1=1 


( 28 ) 


It can be shown that 


Pr[ - — < z < — ] = 1 
Jx\ ^ 


(29) 


(see Appendix B). Therefore, the statistic z, which is nothing more 
than the average of the difference of the random sample patterns and 


the corresponding estimates of their means, may serve as an Indication 
of divergence. 

The restart "monitor" may thus be implemented as follows. If 
the "confidence Interval" condition 



(30) 


is violated, the algorithm should be restarted at that point (n 
should be reset to one). 

The interval In which z must lie is reduced In proportion to 
1 /v / rT. The maximum rate at which a stochastic approximation of a 
quantity may converge to the true value is in proportion to 1/n, the 
harmonic sequence, and still satisfy equation (10) of Chapter II [8]. 
If the y sequence of equations (9) and (11) approaches the harmonic 
sequence in the limiting case, it would also be desirable to reduce 
the confidence Interval around z in proporation to 1/n. However, were 
the interval around z reduced in proporation to 1/n, the probability 
of divergence would no longer remain approximately equal to one, nor 
would it remain constant for each value of n. 

The effect of the divergence criterion developed above is to 
Increase the sensitivity to divergence as much as possible while 
maintaining a constant probability of successfully detecting 
divergence. In particular this technique has the advantage that it 
may be used to monitor any estimation algorithm, no matter what 
degree of complexity was assumed in algorithm derivation. 
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One point of interest concerns the variance of Z. Since accurate 
estimation forms the basis for the confidence interval concept, in- 
accurate estimation will result In the variance associated with Z 

2 

being larger than c . The resulting divergence criterion may be 
stricter than anticipated. This problem may be circumvented by making 
the criterion more lax (for example, by increasing the interval 
length around z to ±4a/^n ). 

An alternative method for testing for divergence would be to 

— 7 

make use of the estimates of mean square error e n made by the algo- 

/ 7 

rithms as discussed in Chapter II. If ye could be considered a 

measure of the error between x„ and the true value e(n) and o a 

n 

measure of the error between y n and the true value e (n ) then x n and y n 

/‘‘"y 

should differ at most by y + o. An algorithm restart, with n reset 
to one, could be made at the point where 



with K a constant factor (for example, K = 2). 

Another possible variation on this idea might be to use both 
the original divergence criterion (confidence interval) together 
with this latter relation in combination. 
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CHAPTER IV 


ADAPTIVE RECOGNITION AND BOUNDARY 
DEFINITION PROGRAM 

An adaptive Bayes classifier is realized by incorporating within 
the ordinary Bayesian classification program estimation operations 
which optimally estimate statistics for the next classification time. 

An application suggested was that the adaptive classifier might be 
useful in locating or defining spatial boundaries (not to be confused 
with the Bayes decision surface or boundary) between data classes. A 
physical example would be the definition of the shoreline between a 
body of water and a land mass; varying means would then correspond to 
spectral shifts of scan data caused by transition from deep water to 
shallow water near the shoreline. As a test, different data sets have 
been generated, each having two equally likely data classes. These 
data sets are composed of patterns synthetically produced to simulate 
a 128 x 128 pixel frame of four dimensional Gaussian spectral scan 
data. 

Adaptive classification and boundary definition programs have 
been developed which treat each of the 128 individual horizontal rows 
as a separate, independent classifier test. These programs utilize 
the CF and the second degree PF algorithms to adapt to changing class 
mean vectors. Updated estimates of the covariance matrix for each 
class are made using the recursive estimation technique discussed in 
Chapter II. 
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A general flow chart of program operation Is shown In Figure 4. 
Program initialization Is accomplished by specifying an appropriate 
disk file of input data for classification, specifying a disk file to 
contain output boundary results for video display, specifying Initial 
estimates of the mean vectors and covariance matrices of the two 
classes, and inputting a decision variable. The process of classifi- 
cation and subsequent boundary definition then begins. 

A 128 pattern row of data is read into memory from the input 
disk file, each pattern of which Is four dimensional. Patterns are 
classified by a Bayesian classification subroutine. The classifier 
returns the variable I CLASS as a one or a two to indicate that the 
pattern has been assigned to class one or class two. 

In order to determine whether or not a boundary between the two 
classes has been crossed in a rov* test, a stack, whose length is 
assigned by the specification of the decision variable at initilizatiori, 
is used. ICLASS associated with the first classified pattern of a 
row is stored and also pushed onto the stack. The value of ICLASS 
associated with each successive classified pattern is pushed onto the 
stack. Only when the stack is full may a decision be made as to 
whether or not a boundary has been crossed. At that time, and sub- 
sequent times, each element of the stack is examined; if more than 
half of the members of the stack have values equal to that of the 
ICLASS of the first classified pattern of the row, the boundary 
definition algorithm decides no boundary has been crossed. If more 
than half of the members of the stack differ from the ICLASS of the 
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first classified pattern of the row, the algorithm decides a boundary 
has been crossed and the value stored for the ICLASS of the first 
classified pattern of the row Is replaced by the ICLASS of the new 
class which has been encountered. The appropriate boundary address 
is stored and the same process continues for the remainder of the 
row. 

After classifying each pattern and performing the boundary test, 
the divergence criterion of Chapter III may be employed to determine 
whether or not the estimates of particular mean vector components have 
diverged. Divergence of a mean vector component requires a restart of 
the estimation algorithm for that component In the area of divergence. 

As each pattern is classified, class statistics for the ap- 
propriate class must be projected ahead for the next classification by 
either the CF or the PF algorithms and the recursive form for the 
covariance estimation. Upon completion of a 128 pattern row test, 
boundary information is written into the disk output file, the next 
row of input data is read, and the process Is begun on the unclassified 
row. 

This procedure is repeated until classification and subsequent 
boundary definition of all 128 rows Is accomplished. Upon completion, 
all Input and output disk files are closed and program execution 
terminates. 

Appendices C and D each contain a compiled Fortran IV program 
listing of two different version of an adaptive Bayes classifier. The 
numbers at the leftmost side of the listings correspond to the internal 
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sequence or statement numbers supplied by the Digital Equipment 
Corporation RT-11 Operating System FORTRAN Compiler. These statement 
numbers will be used in reference to particular statements. 

The first version of the classifier (Appendix C) incorporates 
the modified CF, algorithm to adoptively estimate class mean vectors, 
the confidence interval divergence criterion to test for divergence 
of mean estimates, and the recursive form of covariance estimation. 

In order to adapt to class mean vectors only and check for their 
divergence, the statement corresponding to line 117 of the main program 
should be deleted. To adapt to mean vectors only and neglect the 
possibility of their divergence, statements corresponding to line 
numbers 62 through 115 as well as line 117 of the main program should 
be deleted. To implement the unmodified CF algorithm to adapt mean 
vectors only, statements corresponding to lines 6, 7, 12, 16, 17, and 
22 of SUBROUTINE PROJECT and lines 62 through 115 and also line 117 
of the main program should be deleted. An ordinary Bayes classifier 
(non-adaptive) may be implemented by deletion of lines 62 through 117 
of the main program. 

The second version of the classifier (Appendix D) incorporates 
the second degree PF algorithm to adaptively estimate class mean 
vectors and the recursive form of covariance estimation. In order to 
adapt to class mean vectors only, the statement corresponding to line 
65 should be deleted. To implement an ordinary (non-adaptive) Bayes 
classifier, the statements corresponding to lines 64 and 65 may be 
deleted. 
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CHAPTER V 
RESULTS 

Five data sets have been synthesized to simulate five 128 * 128 
pixel mul tlspectral scan data frames [4]. These data sets are each 
composed of two classes of four dimensional Gaussian data. A photo- 
graph depicting the true spatial boundary between the two classes is 
shown In Figure 5. The area to the left of this wedge shaped boundary 
Is referred to as class one; similarly, the area to the right of the 
boundary Is class two. The shortest and longest rows of data for each 
class are 32 and 96 patterns. 

Individual rows of data were generated a row at a time from 
left to right. Data sets one and two were both generated with all four 
class one mean components varying according to the relation 

ms < N - 32 > 2 + 5 

from the left edge of the frame to the boundary (N is simply the 
position Index having an Initial value of zero at the left edge of 
the frame and Incremented by one at each position to the right). A 
plot of this relation versus N Is shown In Figure 6. Class two data 
was generated for the remainder of each row. Class two of data set 
one was generated having the constant mean vector 
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0 

0 

0 

0 


while the mean vector associate with class two of data set two Is 


"1.5 ' 

1.5 

1.5 

1.5 


The covariance matrices of both classes of data sets one and two are 


“ 1 

.5 

.5 

.5 ‘ 

.5 

1 

.5 

.5 

.5 

.5 

1 

.5 

_ .5 

.5 

.5 

1 _ 


Data set three was generated with the four class one mean com- ^ 

ponents varying according to the relation 

7.5 + 2.5 cos (.1047N) 

from the left edge of the frame to the boundary (N agalr denotes a 
positional Index). A plot of this relation Is shown in Figure 7. 
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Class two data have been generated for the remainder of each row having 
the constant mean vector 


0 

0 

0 

_0 J . 

The covariance of both class one and two of data set three is the same 
as was specified for data sets one and two. 

Data sets four and five were generated having class one and 
two means specified in exactly the same manner as data set one. In 
addition, however, each term of the covariance matrix of the class one 
data v/as changed in a linear manner according to the equation 

c 1j(N) = C 1 j (0) + m N 

1 = 1 , ..., 4 
j = 1 4 

*> 

where m Is simply a slope factor. In other words a linear scalar 
function of position Is added to each term cf the initial covariance. 

For data sets four and five the initial class one covariance was 
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C(0) ■ 


"1 0 
0 1 
0 0 
0 0 


0 0 ’ 
0 0 
1 0 
0 1 


Covariance matrix elements of class one, data set four were varied with 
a slope m of 0.C2 while like elements of data set five changed with a 
slope of 0.2. Covariance matrices for class two of data sets four and 
five were both specified as 


1 0 0 0 ’ 

0 10 0 

0 0 10 

0 0 0 1 


Eight different classification and boundary definition programs 
have been applied to the problem of striking the boundary separating 
the two classes in each of the five data sets. Each program requires 
Initial estimates of the mean vectors and covariance matrices of the 
two classes. Because the estimation algorithms predict well, initial 
mean vector estimates may be made by training over a small area. The 
effect Is to hold sample scatter to a minimum while providing 
reasonable estimates of the mean. 

The first program, referred to as BAYES1 , implements an ordinary, 
non-adaptlve Bayes classifier. The initial estimates of class mean 
vectors and covariance matrices are Incorporated throughout classifi- 
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cation of a complete data set and the resulting data file containing 
boundary Information may be displayed by the DATA-DISK video system. 

The second program, BAYES2, employs the CF algorithm in its 
original form to adaptively estimate mean vectors for a Bayes clas- 
sifier. Boundary data Is subseqjently deduced and stored for display. 

Program number three, BAYEi>3, utilizes the modified CF algorithm 
discussed in Chapter II to proc.jce up-to-date estimates of changing 
mean vectors for a Bayes class' fier. 

BAYES4 incorporates not only the modified CF algorithm, but 
also the confidence Inte val divergence criterion introduced in 
Chapter III to adaptively estimate class mean vectors for the 
classifier. 

BAYES5 implements the second degree PF algorithm to adaptively 
project estimates of class mean vectors for a Bayesian classifier. 
BAYES6, BAYES7, and BAYES8 take the same form as BAYES3, BAYES4, and 
BAYES5, respectively, with the exception that BAYES6 through BAYES8 
also employ the recursive covariance estimation technique. 

Table I provides a cross-reference summary relating Figures 8 
through 47 to the particular data sets and programs. Each figure is 
also individually Identified by the program name and data set number 
used. These figures are photographs of boundaries defined by the 
various programs for each data set. 

A comparison of the results obtained applying the various 
programs -to the different data sets reveals that the ability to adapt 
to changing mean vectors Is essential to successful classification. 
False boundaries have been generated in each case where the non- 
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TABLE I 

A CROSS-REFERENCE OF FIGURES DEPICTING RESULTS OBTAINED 
UPON APPLICATION OF THE CLASSIFICATION AND 
BOUNDARY DEFINITION PROGRAMS TO THE 
VARIOUS DATA SETS 



PROGRAM 

1 

2 

DATA SETS 
3 

4 

5 


BAYES1 

Figure 8 

Figure 16 

Figure 24 

Figure 32 

Figure 40 

BAYES2 

Figure 9 

Figure 17 

Figure 25 

Figure 33 

Figure 41 

BAYES3 

Figure 10 

Figure 18 

Figure 26 

Figure 34 

Figure 42 

BAYES4 

Figure 11 

Figure 19 

Figure 27 

Figi re 35 

Figure 43 

BAYES5 

Figure 12 

Figure 20 

Figure 28 

Figure 36 

Figure 44 

BAYES6 

Figure 13 

Figure 21 

Figure 29 

Figure 37 

Figure 45 

BAYES 7 

Figure 14 

Figure 22 

Figure 30 

Figure 38 

Figure 46 

BAYES8 

Figure 15 

Figure 23 

Figure 31 

Figure 39 

Figure 47 
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Figure 8, Spatial boundaries resulting from the ap- 
plication of an ordinary Bayes classifier (BAYES1) 
to data set 1. Mote the false boundaries. 



Figure 9. Spatial boundaries resulting from the ap- 
plication of an adaptive Bayes classifier (BAYES2) 
using the CF algorithm to data set 1. Note the 
false boundaries. 
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Figure 10. Spatial boundaries resulting from the ap- 
plication of an adaptive Ba/es classifier (BAYES3) 
using the modified CF algorithm to data set 1. 



Figure 11. Spatial boundaries resulting from the ap- 
plication of an adaptive Bayes classifier (BAYES4) 
using the modified CF algorithm and the divergence 
criterion to data set 1. 


I* 
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Figure 12. Spatial boundaries resulting from the ap- 
plication of an adaptive Bayes classifier (BAYESS) 
using the PF algorithm to data set 1. 



Figure 13. Spatial boundaries resulting from the ap- 
plication of an adaptive Bayes classifier (BAYES6) 
using the modified CF algorithm and recursive 
covariance estimation to data set 1. 
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Figure 14. Spatial boundaries resulting from the ap- 
plication of an adaptive Bayes classifier (BAYES7) 
using the modified CF algorithm, divergence criterion, 
and recursive covariance estimation to data set 1. 



Figure 15. Spatial boundaries resulting from the ap- 
plication of an adaptive Bayes classifier (BAYES8) 
using the PF algorithm and recursive covariance 
estimation to data set 1. 
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Figure 16. Spatia. boundaries resulting from the ap- 
plication of an ordinary Bayes classifier (BAYES1) 
to data set 2. Note the false boundaries. 



Figure 17. Spatial boundaries resulting from the ap- 
plication of an adaptive Bayes classifier (BAYES2) 
using the CF algorithm to data set 2. Note the 
false boundaries. 
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Figure 18. Spatial boundaries resulting from the ap- 
plication of an adaptive Bayes classifier (BAYES3) 
using the modified CF algorithm to data set 2. 





Figure 19. Spatial boundaries resulting from the ap- 
plication of an adaptive Bayes classifier (BAYES4) 
using the modified CF algorithm and the divergence 
criterion to data set 2. 
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Figure 20. Spatial boundaries resulting from the ap- 
plication of an adaptive Bayes classifier (BAYCS5) 
using the PF algorithm to data set 2. 





Figure 21. Spatial boundaries resisting from the ap- 
plication of an adaptive Bayes classifier (BAYES6) 
using the modified CF algorithm and recursive 
covariance estimation to data set 2. 


rigure 22. Spatial boundaries resulting from the ap- 
plication of an adaptive Bayes classi "ier (BAYES7) 
using the modified CF algorithm, divergence criterion, 
and recursive covariance estimation to data set 2. 



Figure 23. Spatial boundaries resulting from the ap- 
plication of an adaptive Bayes classifier (BAYESS) 
using the PF algorithm and recursive covariance 
estimation to data set 2. 
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Figure 24. Spatial boundaries resulting trom the ap- 
plication of an ordinary Bayes classifier (BAYES1 ) 
to dati» set 3. Note the false boundaries. 



Figure 25. Spatial boundaries resulting from Lhe ap- 
plication of an adaptive Bayes classifier (BAYES2) 
using the CF algorithm to data set 3. Note the 
false boundaries. 
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Figure 26. Spatial boundaries resulting from the ap- 
plication of an adaptive Bayes classifier (BAYES3) 
using the modified CF algorithm to data set 3. 



Figure 27. Spatial boundaries resulting from the ap- 
plication of an adaptive Bayes classifier (BAYES4) 
using the modified CF algorithm and the divergence 
criterion to data set 3. 



Figure 28. Spatial boundaries resulting froni the ap- 
plication of an adaptive Bayes classifier (BAYES5) 
using the PF algorithm to data set 3. 



Figure 29. Spatial boundaries resulting from the ap- 
plication of an adaptive Bayes classifier (BAYES6) 
using the modified CF algorithm and recursive 
covariance estimation to data set 3. 
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Figure 30. Spatial boundaries resulting from the ap- 
plication of an adaptive Bayes classifier (BAYES7 ) 
using the modified CF algorithm, divergence criterion, 
and recursive covariance estimation to data set 3. 





Figure 31. Spatial boundaries resulting from the ap- 
plication of an adaptive B-.yes classifier (BAYES8) 
using the PF algorithm and recursive covariance 
estimation to data set 3. 



Figure 32. Spatial ! undaries resulting from the ap- 
plication of an ordinary Bayes classifier (BAYES1) 
to data set 4. Note the false boundaries. 



Figure 33. Spatial boundaries resulting from the ap- 
plication of an adaptive Bayes classifier (BAYES2) 
using the CF algorithm to data set 4. Note the 
false boundaries. 
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Figure 34. Spatial boundaries resulting from the ap- 
plication of an adaptive Bayes classifier (BAYES3) 
using the modified CF algorithm to data set 4. 



Figure 35.. Spatial boundaries resulting from the ap- 
plication of an adaptive Bayes classifier (BAYES4) 
using the modified CF algorithm and the divergence 
criterion to data set 4. 



Figure 36. Spatial boundaries resulting from the ap- 
plication of an adaptive Bayes classifier (BAYES 
using the PF algorithm to data set 4. 
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Figure 37. Spatial boundaries resulting from the ap- 
plication of an adaptive Bayes classifier (BAYES6) 
using the modified CF algorithm and recursive 
covariance estimation to data set 4. 


cn 
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Figure 38. Spatial boundaries resulting from the ap- 
plication of an adaptive Bayes classifier (BAYES7) 
using the modified CF algorithm, divergence criterion, 
and recursive covariance estimation to data set 4. 
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Figure 39. Spatial boundaries resulting from the ap- 
plication of an adaptive Bayes classifier (BAYES8) 
using the PF algorithm and recursive covariance 
estimation to data set 4. 
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Figure 40. Spatial boundaries resulting from the ap- 
plication of an ordinary Bayes classifier (BAYES1) 
to data set 5. Note the false boundaries. 








i 





Figure 41. Spatial boundaries resulting from the ap- 
plication of an adaptive Bayes classifier (RAYES2) 
using the CF algorithm to data set 5. Note the 
false boundaries. 
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Figure 42. Spatial boundaries resulting from the ap- 
plication of an adaptive Bayes classifier (BAYES3) 
using the modified CF algorithm to data set 5. 

Note the false boundaries. 



Figure 4 3. Spatial boundaries resulting from the ap- 
plication of an adaptive Bayes classifier (BAYES4) 
using the modified CF algorithm and the divergence 
criterion to data set 5. Note the false boundaries 
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Figure 44. Spatial boundaries resulting from the ap- 
plication of an adaptive Bayes classifier (BAYES5) 
using the PF algorithm to data set 5. Note the 
false boundaries. 



Figure 45. Spatial boundaries resulting from the ap- 
plication of an adaptive Bayes classifier (BAYES6) 
using the modified CF algorithm and recursive 
covariance estimation to data set 5. Note the 
false boundaries. 
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Figure 46. Spatial boundaries resulting from the ap- 
plication of ari adaptive Bayes classifier (BAYES7) 
using the modified CF algorithm, divergence criterion, 
and recursive covariance estimation to data set 5. 

Note the false boundaries. 
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Figure 47. Spatial boundaries resulting from the ap- 
plication of an adaptive Bayes classifier (BAYES8) 
using the PF algorithm and recursive covariance 
estimation to data set 5. Note the false boundaries. 
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adaptive classifier, BAYES1, has been applied to each data set so 
as to obscure the form of the true boundary. No Improvement In the 
boundary definition results In using the original form of the CF 
algorithm. Implemented In BAYES2, as the adaptive estimator of mean 
vectors. 

Significant Improvement In boundary definition performance 
results have been achieved by BAYES3 which employs a modified CF algo- 
rithm to adaptively estimate class means. The modification (see 
Chapter II) consists of subtracting the initial value of the mean 
vector of the classified pattern from the pattern and current mean, 
applying the CF algorithm to the results, and finally adding to the 
projected mean estimate made by the CF algorithm the initial mean 
value. 

Still more improvement resulted upon application of BAYES4 
which incorporates the modified CF algorithm and also the confidence 
Interval divergence criterion of Chapter III. The effect of employing 
this criterion was to restart the modified CF algorithm (as If n were 
1 again) if the condition 


n 

i 

1=1 


(x rV 


< 


3o 


ft 


was violated. Most of the erroneous boundary points appear at points 
where restarts were made, due to poor initial tracking when the algo- 
rlghm is first started with little prior training. 
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Boundary definition resulting from the application of BAYES5 Is 


very satisfactory, especially In the case of data set two where the 
degree of overlap Is large. 

In no case were the results obtained using BAYES6, BAYES7, and 
BAYES8 of equal quality to the results obtained using BAYES3, BAYES4, 
and BAYES5. Even in the case of data set four In which a class 
covariance was changing slowly, the programs which Ignored the fact 
that a covariance matrix could be position dependent proved superior. 

In data set five the covariance associates with class one grew 
at such a rapid rate that class one quickly overlapped class two 
resulting in an Impossible situation for each of the eight techniques. 


CHAPTER VI 


CONCLUDING OBSERVATIONS 

PF type algorithms represent an algorithm class that predicts 
well; a second degree PF algorithm was used as an example In this 
thesis tut algorithms of this class can be derived (with different 

A 

S and y formulas) for tracking parameters that vary with time as an 
n^ degree polynomial. In order to achieve the best results, the 
degree of polynomial assumed in algorithm derivation should be of 
approximately the same order as anticipated variation. A PF type algo- 
rithm can also track variations not of the exact form assumed because 
of the limited memory characteristic of the "refine" step [7]. Another 
advantage of the PF class of algorithms Is that shifts between algo- 
rithms of different complexity can be effected in mid-operation since 
the output of any PF algorithm (the error estimate and projected 
parameter estimate) along with the next data sample may serve as the 
input to any other algorithm of the PF form. 

Modifications of the CF algorithm have also been found suitable 
for tracking varying parameters. Although a PF algorithm may produce 
better estimates than modified versions of the CF algorithm, especially 
as the degree of class overlap is increased, the additional memory 
required for storage of past history may In some cases prohibit use of 
the PF form and warrant utilization of the modified CF algorithm. 

Much work has been done In the area of state estimation In 
controls engineering. Kalman predictors are capable of producing 
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statistically optimum state estimates when measurements and inputs are 
stochastic In nature [12,13]. The Kalman predictor has been found 
similar in nature to estimation algorithms presented in this thesis. 

An area for future study lies In investigating the possibility of 
modifying the Kalman preJictor to account for changing covariance. 
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APPENDICES 


APPENDIX A 


COVARIANCE ESTIMATION 

Letting C(N) represent the estimate of the covariance for N 
samples. 


N 

C (N) = Z y i y j 
j-l 

where the expected value of y has been approximated by the sample 
average m(N) [1]. 


N+l 

C(N+1 ) = ^ l yj y / - m(N+l ) nT(N+l) 

J-l 


N 

= NTT ( l y j y / + y N+l y N+l } ' m(N+1) m (N+1) 

J-l 


- (NC(N) + N m(N) nT(N) + y N+1 y' N+] ) 


■ — 5" (N m(N) + X ,)(N m(N) + X N+1 )' 

(N+l j N 1 N 1 
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This expression provides a convenient method for estimating or 
updating the covariance matrix, starting with C(1) = y 1 y 1 - m(l) m (1). 
Since m(l) = y^ , C(1) = 0, the zero matrix. 


APPENDIX B 


CONFIDENCE INTERVAL DERIVATION 
Consider the statistic 


z 


n 

~ I 

n L 
1=1 


If z denotes the mean of a random sample of size n from a distribution 
Z “ N( u z , a 2 ) , then z “ N(u z> o 2 /n) [10,11]. Consider the probability 
that the interval (u z - 3 o/*^, u z + 3a/ v^) includes the point z. The 
event u z - 3a//rT < z < u 2 + 3 o/vfr occurs when and only when the event 
-3 < » / n’(u z - ~z)/o < 3 occurs, thus these two events have the same 
probability. However, v^n^ - "z)/o Is N(0,1). Accordingly, the 
probability that the interval (u z - 3o//n, u z + 3a//n) includes the 
point z is equal to 


3 2 

J- e” z /2 dz = 0.998 . 

Ztt 

-3 

_ 2 

This probability in no manner depends upon the values of z, o , or the 

Integer n. Consider next, the length of the Interval. The length is 

2 

seen to be 6o/*^n. Note that this length is unknown until both c and 
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? 

n are known; note also that for o assigned, this length may be made 
as short as desired by taking n sufficiently large. 

For the particular interval considered above, 0.998 was found 
to be the probability that the interval (u z - 3o/*^n, u ? + 3o/^) 
contains the statistic 7 . That Is, 

Pr[u z - 3c/*^n < z < u z + 3o/*^n ] = 0.998 . 

Since a Is known, each of the variables u z - 3c/» / n and u z + 3o/»^n is 
a known quantity if u z Is known. 
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IY -VERTICAL PICTURE ARRAY INDEX 
ICLASS -EITHER 1 OR 2 INDICATING WHETHER LAST PATTERN 
WAS CLASSIFIED A MEMBER OF CLASS ONE OR TWO 
INDEX -AN INDEX OF THE IADATA ARRAY 


DATAI -ARRAY TO RECEIVE 128 FOUR DIMENSIONAL DATA POINTS 
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APPROXIMATION OF CLASS TWO MEAN VECTOR 
CIA -ARRAY CONTAINING CONVERGENCE INFORMATION FOR USE IN 
STOCHASTIC APPROXIMATION OF CLASS ONE MEAN 
CIB -ARRAY CONTAINING CONVERGENCE INFORMATION FOR USE IN 
STOCHASTIC APPROXIMATION OF CLASS TWO MEAN 


IADATA -ARRAY CONTAINING ROW BOUNDARY POSITIONS 
ICDATA -A LOGICAL STAR*! ARRAY USED IN BUILDING A DISK OUTPUT 
FILE FOR DISPLAY VIA DATA-DISK VIDEO SYSTEM 
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0062 CALL AVERAGE (X,U1 ,U2,U1BAR,U2BAR,ICLASS,TIME1 ,TIME2 

0063 CALL INTERVAL (TIME1 ,TIME2,ICLASS,CIA,CIB) 

0064 CI=FL0AT(IC) 

0065 I F( I CLASS . EQ . 2 ) GO TO 34 

0C67 I F (TIME! (1 J.LE.CI) GO TO 80 
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0119 DO 91 M=1 ,128 

0120 IODATA(M)=IBLNK 
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0001 SUBROUTINE INPUT (IY) 

C THIS SUBROUTINE READS ONE ROW OF FOUR-SIMENSIONAL DATA OF A 128 X 128 
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0001 SUBROUTINE DECIDF(X,CAI ,CBI ,UA,UB,DA,DB ,ICLASS) 

C THIS SUBROUTINE IMPLEMENTS A BAYES CLASSIFIER FOR TWO CLASSES 
C OF EQUAL A PRIORI PROBABILITY. 
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0001 SUBROUTINE BOUND(IDATA,ICLASS, IX, IC, INDEX) 

C THIS SUBROUTINE FORMS A BOUNDARY BETWEEN THE TWO DATA CLASSES. 

0002 DIMENSION IDATA(128) ,IC0N(32) 
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0001 SUBROUTINE AVERAGE (X,U1 ,U2, XI BAR, X2BAR, I CLASS, TIMEX, TIMEY) 

C THIS SUBROUTINE KEEPS AN INDEPENDENT RUNNING TIME AVERAGE OF THE 
C DEVIATION OF THE DATA FROM THE TRUE OR PROJECTED MEAN OF EACH 
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0001 SUBROUTINE INTERVAL(TIMEX,TIMEY,ICLASS,CIA,CIB) 

C THIS SUBROUTINE CALCULATES THE CONFIDENCE INTERVAL ASSOCIATED WITH 
C EACH COMPONENT OF THE MEAN VECTOR FOR THE CLASS OF INTEREST. 

C THE CONFIDENCE INTERVAL IS FOR USE AS AN INDICATION OF DIVERGENCE. 
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0001 SUBROUTINE PROJECT(X,UX,UY, ICLASS,UA,UB, TIMEX, TIMEY) 

C THIS SUBROUTINE PROJECTS EACH COMPONENT OF THE MEAN VECTOR OF THE 
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0001 SUBROUTINE PROCOV(CA,CB,CAINV,CBINV,DA,DB,X,ICLASS,IC,IFLAG) 

C THIS SUBROUTINE MAKES A NEW ESTIMATE OF THE COVARIANCE OF A 
C CLASS AFTER IC POINTS HAVE BEEN CLASSIFIED INTO THAT PARTICULAR 
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000000000000000000000000 


0031 DO 40 1 = 1 ,16 

0032 CB(I)=PHIB(I) 

0033 C3INV( I )=PHIB(I) 


0034 40 CONTINUE 

0035 CALL MI NV ( CBI NV ,4,DB) 
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0001 SUBROUTINE COVAR(COV,X, INDEX, NDIM) 

C THIS SUBROUTINE IMPLEMENTS THE RECURSIVE FORM OF COVARIANCE 
C ESTIMATION. 
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0001 SUBROUTINE MINV(A,N,D) 

C THIS SUBROUTINE FINDS THE INVERSE OF A GENERAL MATRIX 
C 'A', WHICH IS DESTROYED IN COMPUTATION. THE INVERSE 
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0C24 KI=KI+M 

."?025 H0LD=-A(KI ) 

0026 JI=KI-K+J 

0027 A(KI )=A(JI ) 

0028 30 A( JI )=H0LD 


0029 35 I=M(K) 

0030 IF(I-K)45,45,38 

0031 38 JP=N*(I-1 ) 
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00E3 KJ=KJ+N 

0060 IF(J-K) 70,75,70 

0061 70 A(KJ)=A(KJ)/BIGA 

0062 75 cormnuE 


0063 D= 0*13 IGA 

0064 A(KK)=1 .0/BIGA 
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0001 SUBROUTINE OUTPUT (IY) 

C THIS SUBROUTINE WRITES ONE 128 POINT ROW (BYTE DATA POINTS) OF A 
C 128 X 128 POINT ARRAY FOR BOUNDARY DISPLAY VIA VIDEO DISPLAY SYSTEM. 
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0005 COMMON /DUMB1/DATAI 

0006 COMMON /DUMB2/ICDATA 
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0061 X(4)=DATAI (J+3) 

0062 CALL DECI DE ( X , Cl I ,C2I ,U1 ,U2,D1 ,DZ,ICLASS) 

0063 CALL BOUND ( I ADATA , I CLASS , I X , I C , I NDEX ) 

0064 88 CALL PR0JECT(X,U1 ,U2, ICLASS.El ,E2, TIME1 ,TIME2) 


0065 CAL L PR0C0V ( Cl , C2 , Cl I , C2 1 , 01 , D2 , X , I CL ASS , I C , 1 ) 

0056 90 CONTINUE 

0067 DO 91 M=1 ,128 
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0001 SUBROUTINE INPUT (IY) 

C THIS SUBROUTINE READS ONE ROW OF FOUR-DIMENSIONAL DATA OF A 128 X 128 
C DATA POINT ARRAY FROM A PRESPECIFIED DISK FILE. THE ROW OF DATA 
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0001 SUBROUTINE DECIDE(X,CAI ,CBI ,UA,UB,DA,DB,ICLASS) 

C THIS SUBROUTINE IMPLEMENTS A BAYES CLASSIFIER FOR TWO CLASSES 
C OF EQUAL A PRIORI PROBABILITY. 
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0001 SUBROUTINE BOUND(IDATA,ICLASS, IX, IC, INDEX) 

C THIS SUBROUTINE FORMS A BOUNDARY BETWEEN TKE TWO DATA CLASSES 
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0001 SUBROUTINE PROJECT(X, UX,UY,ICLASS, El ,E2, TIMEX, TIMEY) 

C THIS SUBROUTINE PROJECTS EACH COMPONENT OF THE MEAN VECTOR OF THE 
C CLASS OF INTEREST. THE PROJECTION IS MADE USING A STOCHASTIC 
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0032 X2(J,I)=X(I) 

0033 IF(TIMEY(I).LT.3.)G0 TO 45 

0035 II=IFIX(TIMEY(I)/2. )+l 
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0001 SUBROUTINE PROCOV(CA,CB,CAINV,CBINV,DA,DB,X,ICLASS,IC,IFLAG) 

C THIS SUBROUTINE MAKES A NEW ESTIMATE OF THE COVARIANCE OF A 
C CLASS AFTER IC POINTS HAVE BEEN CLASSIFIED INTO THAT PARTICULAR 
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0031 DO 40 1=1,16 

0032 CB(I)-PHIB(I) 

0033 CB I NV ( I )=PHIB(I ) 

0034 40 CONTINUE 

0035 CALL MINV(CBINV,4,DB) 


0036 ICNTB=1 

0037 50 RETURN 

0038 END 
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0001 SUBROUTINE COVAR(COV,X, INDEX, NDIM) 

C THIS SUBROUTINE IMPLEMENTS THE RECURSIVE FORM OF COVARIANCE 
C ESTIMATION. 




0001 SUBROUTINE MINV(A,N,D) 

C THIS SUBROUTINE FINDS THE INVERSE OF A GENERAL MATRIX 
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0064 A(KK)=1 . 0/BI6A 

0065 80 CONTINUE 
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0001 SUBROUTINE MPRD(A,B,R,N,M,L) 

C THIS SUBROUTINE FORMS THE PRODUCT OF TWO GENERAL MATRICES 
C 'A' AND 'B\ AND RETURNS THE PRODUCT IN ' R' . ' N' IS THE NUMBER 

C OF ROWS IN 'A', ' M' IS THE NUMBER OF COLUMNS IN 'A' AND ROWS 
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0001 SUBROUTINE OUTPUT (IY) 

C THIS SUBROUTINE WRITES ONE 128 POINT TOW (BYTE DATA POINTS) OF A 




